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ON THE DISTRIBUTION OF THE TRACE 
IN THE UNITARY SYMPLECTIC GROUP 
AND THE DISTRIBUTION OF FROBENIUS 

GILLES LACHAUD 


Abstract. The purpose of this article is to study the distribution of the trace 
on the unitary symplectic group. We recall its relevance to equidistribution 
results for the eigenvalues of the Frobenius in families of abelian varieties over 
finite fields, and to the limiting distribution of the number of points of curves. 
We give four expressions of the trace distribution if g = 2, in terms of special 
functions, and also an expression of the distribution of the trace in terms of 
elementary symmetric functions. In an appendix, we prove a formula for the 
trace of the exterior power of the identity representation. 
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1. Introduction 

Let G be a connected compact Lie group, and 7r : G —> GL(V) a continuous 
representation of G on a finite dimensional complex vector space V. The map 

m i —y T(m) = Trace 7r(m) 

is a continuous central function on G, whose values lie in a compact interval 1 Cl. 
The distribution or law of r is the measure = r*(dm) on / which is the image 
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by r of the mass one Haar measure dm on G. That is, for any continuous real 
function ip £ ^(J), we impose the integration formula 


Alternately, if a; £ 


J <p(x)ht(x) 

then 


<^(Trace7r(m)) dm. 


r 

volume {m £ G | Trace n(m) < x} = / g T . 

J — OO 


We are especially interested here with the group G = USp 2g of symplectic 
unitary matrices of order 2 g, with 7 r equal to the identity representation in V = C 4 . 
With the help of Weyl’s integration formula, one establishes that the distribution 
has a density f T1 that is, a positive continuous function such that dg T (x) = f T (x)dx. 
Our main purpose is the study of / x , especially in the case g = 2 and g = 3. For 
instance, for g = 2, we have 


f T ( x) =- 1-- 


47r 


16 


2F1 


3 U-l-Z- 

2’2’ ’ 16 


if |a"| < 4, with Gauss' hypergeometric function 2 -Pi (see Theorem 5.2). 

Another representation of the distribution of the trace, following a program of 
Kohel, is realized by the Viete map , which is the polynomial mapping 


s(<) = ..., s g (t)), t = (ti,..., t g ), 

where s n (t) is the elementary symmetric polynomial of degree n. Let I g = [—2, 2] s . 
The symmetric alcove is the set 


£ g = s (Ig) c R s , 

which is homeomorphic to the g -dimensional simplex. By a change of variables in 
Weyl’s integration formula, one obtains a measure a x on the hyperplane section 


such that, if | x\ < 2g, 


V X = {S £ Eg | si = x} 


It(x) = / a x (s). 

Jv„ 


As a motivation for the study of these distributions, it is worthwhile to recall that 
they provide an answer to the following question: 

Can one predict the number of points of a curve of given genus over a finite field? 
When a curve C runs over the set M 9 (F ? ) of Fg-isomorphism classes of (nonsingular, 
absolutely irreducible) curves of genus g over F g , the number |C(F 9 )| seems to 
vary at random. According to Weil’s inequality, an accurate approximation to this 
number is close to q + 1, with a normalized “error” term t(C) such that 

|C(F 9 )| = (? + 1- (Z 1 / 2 t(C), \r(C)\<2g. 

The random matrix model developed by Katz and Sarnak gives many informations 
on the behaviour of the distribution of t(C) on the set M 9 (F 9 ). For instance, 
according to their theory, and letting g be fixed, for every x £ R, we have, as 
q —^ 00 (cf. Corollary 4.3): 

\{C £ Mg(Fg) I t{C)<x}\ 


|M S (Fg)| 


[ f T (s)ds + O (q- 1/2 ) 

J —OO ^ ' 
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Hence, the knowledge of f T provides a precise information on the behaviour of the 
distribution of the number of points of curves. 

The outline of this paper is as follows. After Section [2] devoted to notation, 
we recall in Section [3] the Weyl’s integration formula, expressed firstly in terms 
of the angles (9±... ,9 g ) defining a conjugacy class, and secondly in terms of the 
coefficients tj = 2 cos Qj. We discuss equidistribution results for a family of curves or 
abelian varieties over a finite field in Section [4] In Section [5] we obtain four explicit 
formulas for the trace distribution if g = 2, respectively in terms of hypergeometric 
series, of Legendre functions, of elliptic integrals, and of Meijer G-functions. We 
also give the distribution of the trace for the representation of the group SU 2 x SU 2 
in USp 4 . 

In the second part of the paper, we take on a different point of view by using ele¬ 
mentary symmetric polynomials, and obtain a new expression of Weyl’s integration 
formula. Section [6] defines the Viete map, asssociating to a sequence of coordinates 
the coefficients of the polynomial admitting as roots the elements of this sequence, 
and Section [7] describes the symmetric alcove, that is, the image of the set of “nor¬ 
malized real Weil polynomials” by the Viete map. By a change of variables using 
the Viete map, we obtain in Section [8] a new integration formula, which leads to 
another expression for the distribution of the trace on the conjugacy classes, in the 
cases g = 2 and <7 = 3. If g = 2, we compute also the trace of A 2 7t. Finally, we 
include an appendix on the character ring of USp 2g , including a formula on the 
exterior powers of the identity representation. 

I would like to thank David Kohel for fruitful conversations. Also, I warmly 
thank the anonymous referee for carefully reading this work and for its suggestions, 
especially regarding the appendix. 


2. The unitary symplectic group 


The unitary symplectic group G — USp 2g of order 2 g is the real Lie group of 
complex symplectic matrices 

G = {m € GL 2g (C) | Vn.J.ro = J and Vn.m = la,/} , 

with 



Alternately, the elements of G are the matrices 

m = “ 6 )eSU 2g , a, 6 G M g (C). 


The torus T s = (K/27rZ) ff is embedded into G by the homomorphism 


( 2 . 1 ) 


e = (e 1 ,...,9 g )^h{0) 


(e i01 


0 


V 


0 



0 


0 


\ 

0 

e -i8 g J 


whose image T is a maximal torus in G. The Weyl group W of (G, T) is the semi- 
direct product of the symmetric group & g in g letters, operating by permutations on 
the Oj, and of the group N of order 2 9 generated by the involutions 9j i-A —Qj. Since 
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every element of USp 2g has eigenvalues consisting of g pairs of complex conjugate 
numbers of absolute value one, the quotient T/W can be identified with the set 
Cl G of conjugacy classes of G, leading to a homeomorphism 

T 9 /W ———> T/W ———> C1G 

Remark 2.1. Here is a simple description of the set C1G. Let $ 2 g be the subset of 
monic polynomials p £ M[u] of degree 2 g, with p(0) = 1, with roots consisting of g 
pairs of complex conjugate numbers of absolute value one. If 0 £ T®, let 

Pe{v) = - e l6j )(u - 

j =1 

The map 6 K > pg is a bijection from T B /W to $ 2 g - Renumbering, we may assume 
that 

0 < 0 g < 6»g_! < • • • < 01 < 7T. 

The map m 1 —> cp m {u) = det(w. I — m) induces a homeomorphism 

C1G -—- > $ 2 g 

with cp m = pg if and only if m is conjugate to h( 6 ). The polynomial pg is palin¬ 
dromic , that is, if 

2 g 

P0(U) = Y,(-I) n *n(e)u n , 

n—0 

then a 2 g- n ( 6 ) = a n ( 0 ) for 0 < n < g. 


3. Weyl’s integration formula 


The box X g = [0, n ] 9 is a fundamental domain for N in T® and the map F ha Foh 
defines an isomorphism 


(3.1) 


^(G) c 


—— > ^(x ff ) s y m 

from the vector space ^(G) 0 = ^(CIG) of complex central continuous functions on 
G to the space ^f(X g ) sym of complex symmetric continuous functions on X g . Notice 
that the isomorphism (3.1) has an algebraic analog, namely the isomorphism ( |A.l ) 
in the appendix. Let dm be the Haar measure of volume 1 on G. If F € ^(G) 0 , 
then 


' Cl G 


F {m) dm = / F (m)dm, 

Jg 


where dm is the image measure on Cl G of the measure dm. The following result 
is classical [H Ch. 9, § 6, Th. 1, p. 337], gOl 5.0.4, p. 107], 

Theorem 3.1 (Weyl integration formula, I). If F € ^(G)°, then 

[ F (m) dm = I F o h(9))fj, g (6), 

Jg Jx„ 


with the Weyl measure 


p g { 6 ) = S g ( 6 )dd, dd = d 6 1 ... dd gi 
a 


6g(0) = — f — j (sin0j) 2 (2cos0fe — 2cos 6j) 2 . □ 

9 ' i =1 j<k 
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We call the open simplex 

(3.2) A g = {(d 1 ,...,O g )£X g | 0<9 g <9 g _ 1 <---<e 1 <TT} 

the fundamental alcove in X g . The closure A g of A g is a fundamental domain for 
© g in X g , and, for every / G ^(X- g ) sym , we have 


ix, 


f{9) d6 = g\ f f(9) d9. 
J A„ 


There is another way to state Weyl’s integration formula, which will be used in 
Section [8j Let I g = [—2,2]®. The map 

(0i ,9 g ) i-a (2cos0i,..., 2cos0 s ) 

defines an homeomorphism X g — > I g . Let 

k(ti ,..., t g ) = h ( arccos ..., 




i —,..., arccos — 
2 2 


where h{9) is given by (2.1). Then the map F K > F o k defines an isomorphism 

tf{G)° ———> 

where < ^’(/g) sym is the space of complex symmetric continuous function on I g . For 
an algebraic analog, see the isomorphism (A.2) in the appendix. Let 

a 

(3.3) D 0 (t) = l[(t k -t J ) 2 , Di(t) = n(4-^ 

j<k j—1 

Proposition 3.2 (Weyl integration formula, II). If F £ G)°, then 


F (m)dm= j F o k(t)X g {t) dt, 


where t = (t \,..., t g ) and dt = dt\... dt g , with the Weyl measure 

A g (t)dt, dt = dti... dtg, 

Xg{t) = j2^\ Do{t) ^^ y 

Proof. If if G e £(I g ), we have 

/ ip(2cos 9i, ... ,2cos 9 g )Sg(9) d9 = / ip(t)\ g (t)dt. 
Jx g Ji g 

Apply Weyl’s integration formula of Theorem |3.1| 


□ 


As in (3.2), we call the open simplex 

(3.4) Ag = {t£l g I -2 < ii < t 2 < ■ ■ ■ < tg < 2} 

the fundamental alcove in Then A„ is a fundamental domain of I„ for and 
if / G tf(I g ) sym , we have 

(3.5) 


[ f(t) dt = g\ [ f(t) dt. 
Jin J A<] 
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Examples 3.3. We have 

Mt) = ^(<1 - t 2 ) 2 Y / (4-^)(4-t2). 

The maximum of A 2 in A 2 is attained at the point 

t 0 = (~V2, y/2), with A 2 (t 0 ) = . 

We have also 

A 3(0 = 4^3 (*i - < 2 ) 2 (ti - t 3 ) 2 (t 2 - ^a) 2 \/( 4 — t ?)( 4 — * 1)( 4 — i§). 

The maximum of A 3 in A 3 is attained at the point 

to = (—a/3, 0 , VS), with A 3 (t 0 ) = ^3 . 

Now, for the convenience of the reader, we recall some notation on the distri¬ 
bution of central functions. Let G be a connected compact Lie group. The Haar 
measure dm of volume 1 on G is a probability measure, and G becomes a proba¬ 
bility space; ipso facto , its elements become random matrices, and the functions in 
G)° are complex random variables on G. If F G ^(G)° is a real random variable, 
whose values lie in the compact interval Icl, the distribution or law of F is the 
image measure /./,f = F* dm on I such that 

J ip(x) Pf(x) = j (p(F(m))dm if ip G &(!), 

If B is a borelian subset of /, then 

Up(B) = volume {m € G | F(m) € B} , 


and the cumulative distribution function of F is 

4> F (a;) = P(F < x) = f p F (t) = 

J — OO 

The characteristic function of F is the Fourier transform of p F (x): 

f F {t) = f e itx p F (x) = [ e itHm) dm= [ e ltFoh{9) p g (9). 

J- OO JG JXg 

This is an entire analytic function of f, of exponential type, bounded on the real 
line. The distribution p F has a density if p F (x) = f F {x)dx with a positive function 
/f in !/(/). If p F has a density, and if Fourier inversion holds, then 


/ dm. 

J F (m)<x 


/f(z) 


1 

27r 



f F (t)e~ itx dt. 


Conversely, if f F G L 1 (K), then p F has a density. If G = USp 2g , notice that Weyl’s 
integration formulas supply the joint probability density function for the random 
variables (9 1 ,..., 9 g ) and (fj,..., t g ). 

The distribution p F is characterized by the sequence of its moments 


M„(F) = J x n p F (x) = J F {m) n dm, n > 1 , 

and the characteristic function is a generating function for the moments: 
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Remark 3.4. If 7 r is an irreducible representation of G, with real character then 
the random variable r, is standardized , i.e. the first moment (the mean) is equal 
to zero and the second moment (the variance) is equal to one. 


Remark 3.5. Under suitable conditions, an expression of the density by integration 
along the fibers can be given. For instance, let G = USp 2g , let F € ^(G)° be a 
G°° function, and put J = F o h(U), where U is the open box ] 0 , 7 r[ 9 . If F o h is a 
submersion on U, and if a; £ J, then 

V x = {9 £ U | F o h(0) = x} 


is a hypersurface. Let a x be the Gelfand-Leray differential form on V x , defined by 
the relation 

d {F o h) A ot x = 5g(9)d8i A • • • A dO g . 

For instance, 


a x = (—1)- 7_1 (9(F o h)/d0 j )~ 1 8 g (e)de 1 A... d9 j _ 1 A d9 j+1 ■■■ A dO g 

if the involved partial derivative is 7 ^ 0. Then the distribution is computed by 
slicing: since the cumulative distribution function is 

ds / a s (6), 

Jv s 


$1 


(x) = f A g{0) = f 

J Foh{6)<x J-c 


we have 

fp(x) = [ a x {8). 

Jv x 

See {TJ Lemma 7.2] and [T7] Lemma 8.5]. 


4. Equidistribution 


Let A be an abelian variety of dimension g over F q . The Weil polynomial of A 
is the characteristic polynomial L(A, u ) = det(u. I —Fa) of the Frobenius endomor¬ 
phism Fa of A, and the unitarized Weil polynomial of A is 


L(A,u) = L(A,q 1 / 2 u) = JJ(u — e l9j )(it — e l6j ). 

j =1 


This polynomial has coefficients in Z, belongs to the set $ 2 g defined in Remark 2.1 
and 9{A) = {9\, ... 9 g ) is the sequence of Frobenius angles of A. We write 


2 g 

L(4,«) = ^(-1)\(A)«", 

ro— 0 

keeping in mind that a, 2 g -n{A) = a n (A) for 0 < n < 2 g, since L(A,u) £ $ 2 S . By 
associating to A the polynomial L(A : u), each abelian variety defines, as explained 
in Section [2j a unique class rh(A) in C1G, such that 

L{A,u) = det(zil— m(A)). 


Let A g (F g ) be the finite set of fc-isomorphism classes of principally polarized 
abelian varieties of dimension g over k. The following question naturally arises: 

As q -A 00 , and as A runs over A g (F g ), what are the limiting distributions of the 

random variables ai,... ,a g ? 
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In order to clarify this sentence, we look in particular to the coefficient aq, and 
focus on the Jacobians of curves. Let G be a (nonsingular, absolutely irreducible, 
projective) curve over F g . The Weil polynomial L(C,u) of C is the Weil polyno¬ 
mial of its Jacobian, and similarly for the unitarized Weil polynomial L(C,u), the 
sequence of Frobenius angles 0(C) , the coefficients a n (C), and the conjugacy class 
m(C). If Fq is the geometric Frobenius of G, then 

L(C , u) = det(u. I —q~ 1 l 2 Fc) = det(itl — rh(C)). 


Then 


(4.1) \C(V q )\=q + l-q 1 / 2 T(C), 

where t(C ) = ai(G), namely 

a 

t(C ) = q- 1 / 2 Trace Fc = 2 ^ cos Oj , 

i =i 

with 0(C) = (0 1 ,...,0 g ). 

Then Katz-Sarnak theory sTOj models the behavior of the Weil polynomial of a 
random curve C of genus g over F g by postulating that when q is large, the class 
m(C) behaves like a random conjugacy class in Cl G, viewed as a probability space, 
endowed with the image dm of the mass one Haar measure. Here is an illustration 
of their results. Let R(G ) be the character ring of G (cf. the appendix) and 

f(G)° = R(G) ® C ~ C[2cos0i,... 2cos(9 g ] sym 


the algebra of continuous representative central functions on G, the isomorphism 
coming from Proposition A.l This algebra is dense in ^(G)°, hence, suitable for 


testing equidistribution on C1G. We use the following notation for the average of 
a complex function / defined over a finite set Z: 

1 


f(z)dz = TWT 


z ez 


For every finite field fc, we denote by M g (fc) the finite set of fc-isomorphism classes 
of curves of genus g over fc. The following theorem follows directly, if g > 3, from 
uni Th. 10.7.15] (with a proof based on universal families of curves with a 3 1\ 
structure), and from TO) Th. 10.8.2] if g < 2 (with a proof based on universal 
families of hyperelliptic curves). 


Theorem 4.1 (Katz-Sarnak). Assume g > 1. IfC runs over M g (F g ), the conjugacy 
classes rh(C) become equidistributed in C1G with respect to dm as q —>■ oo. More 
precisely, if F £ iZ(G)°, then 



F (m(C))dC 



F(m) dm + O 



Theorem |4.1| means that the counting measures 


□ 


defined on C1G, converges to dm in the weak topology of measures when q —> oo. 
Since 


F o h(0(C)) = F(m(G)), 




TRACE IN THE UNITARY SYMPLECTIC GROUP 


9 


this theorem means also that if C runs over M s (F g ), the vectors 9(C) become 
equidistributed in the fundamental alcove with respect to the Weyl measure when 
q —> oo. 


Remark 4.2. In the preceding theorem, and the above comments, one can substitute 
the set A 9 (F g ) to the set M g (F g ) Th. 11.3.10]. This is an answer to the question 
raised in the beginning of this section. 


As discussed above, the random variable r(C) rules the number of points on the 
set M g (F 9 ), and its law is the counting measure on the closed interval [~2g,2g]: 

1 29 
l ' 9 ’ q = M (F )l = 

1 gK q> ' C£M 9 (F,) s=-2 g 

where 6m is the Dirac measure at x , with the probability mass function 

, |{Ce M g (F ? ) I (t(C) = x}\ 

Js ’ q[ } |M ff (F,)| 

defined if a; € [—2g,2g\ and q l ^ 2 x £ Z. We put now 


9 

r(m) = Trace to, r o h{6) = 2 cos 9j , 

i=i 

for to £ G and 9 £ X g . We take F(to) = r(m) in Theorem |4.1| and call g T be the 
distribution of the central function t as defined at the end of Section |3] We obtain: 


Corollary 4.3. If q —► oo, the distributions v g , q of the Frobenius traces converge 
to the distribution /j t . More precisely, for any continuous function ip on [—2g, 2g], 
we have 

rig 


<£ ip(r(C))dC = [ ip(x)fj, T (x) + 0 (q 1/2 ) , 

J-2q V ' 

T Ms)ds + O (q- 1 / 2 ) . □ 

J —OO ' ' 


/M 3 (F 9 ) 

and for every we have 

\{C € M g (F g ) | t(C)<x}\ 

|M 9 (F,)| 

Lemma 4.4. If 1 < n < 2g — 1, 

(f a n (A)dA = e n + 0 (q~ 1/2 ') 
■m 8 (f„) v y 

where e n = 1 if n is even and e n = 0 if n is odd. 


Proof. As Equation (A.4) in the appendix, let 

r„(m) = Trace(A" to), 


in such a way that ri = r. By equality (A.5), we have 

a n (A) = r n o h(9(A)). 

Since r„ £ £6(G)°, we have, by Remark [Q| 


® r n o h(9(A)) dA = j T n (m)dm + o(q 1 / 2 ') 
J Ag(F ? ) Jg V J 
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but Lemma |A.6| implies that the multiplicity of the character To of the unit repre¬ 
sentation 1 is equal to e n , hence, 


/ T n (m)dm = £ n . □ 

Jg 


Corollary 4.5. Let u £ C and q —> oo. 
(i) If |it| < q 1 / 2 , then 


® L(A, u ) dA = 


u 2g+2 — q 9+1 
u 2 — q 


o 


(V-*). 


(ii) We have 


(iii) We have 


l \A(V q )\dA = q g +0(q a ~ 1 ) . 

•'A a (F 9 ) 


|C(F g )| dC = q + 0{1). 

J m 3 (f 9 ) 

The implied constants depend only on g. 

Proof. We have 

2g 

L(A,u) = q 9 L (A,q-V 2 u) = £(-l ) n a n (A)q^ g - n ^ 2 u n , 

n—0 


with ao = 1 and a 2 g - n = o, n for 0 < n < g. From Lemma 4.4 we get 

; / a n (A) dA = £ n q^ 2g - n ^ 2 u n + u n O LVa-n-V/A 
J f„(f„) v ' 


q (2g-n)/2 u n 


for 1 < n < 2g — 1 , and there is no second term in the right hand side if n = 0 and 
n = 2 g. Now, if |uj < g 1 / 2 , 


29 „,2g+2 n g +1 

x;^ (2fl - n)/ v = u _ q 

n—0 


u 2 — q 


and the absolute value of the difference between this expression and 

® L(A,u)dA 


is bounded by 


2g—l 


B \u\ n q (2g - n ~ 1)/2 , 


with B depending only on g. If |u| < q 1 / 2 , then \u\ n q < ' 2g n x ^ 2 < q( 2g x )/ 2 , and 
this proves Q. If |u| < 1, then \u\ n q^ 2a ~ n ~ l ' > / 2 < g 9_1 , hence, 


(£ L(A,u)dA = q 9 + 0{q g ~ l ) . 


' A S (F, 

and this proves since |A(F g )| = L(A, 1). Since Lemma pi~~i] holds by substituting 
M a to A g , ([m]) is a consequence of this lemma applied to ai(C), and of formula 

dm □ 
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With Corollary 4.5 I]) , it appears as though the Frobenius angles were close in 
the mean to the vertices of the regular polygon with ( 2 g + 2 ) vertices, inscribed in 
the circle of radius q 1 / 2 , the points ±q 1 / 2 being excluded. 


t 


C 


1%,2 bg,3 hg,5 .. • hg,/j 


-► 



Vertical 


Horizontal 


2 3 


5 


► 


P 


Figure 1. Horizontal versus vertical distribution. 


Another approach on the limiting equidistribution of Frobenius angles is the 
generalized Sato-Tate conjecture, see m for a comprehensive description. Let C 
be an absolutely irreducible nonsingular projective curve of genus g over Q, and S 
a finite subset of prime numbers such that the reduction C p = C\w p over F p is good 
if p £ S. Then one says that the group USp 2ff arises as the Sato-Tate group of C if 
the conjugacy classes fn(C p ) are equidistributed with respect to the Weyl measure 
of G when p —>■ oo. In other words, this means that if F G ‘if (G)°, then 

lim ® F (fn{C p ))dp= / F(m)dm, 

n ^°°JPs(n) Jg 

where Ps{n) = {p £ P\S \ p < n}. The case g = 1 is the Sato-Tate original 
conjecture, now a theorem. Here is an example of what one expects m ■ 

Conjecture 4.6 (Kedlaya-Sutherland). //Endc(JacC) = Z, and if g is odd, or 
g = 2, or g = 6, then the group USp 2g arises as the Sato-Tate group of C. 

The two preceding types of equidistribution are symbolically shown in Figure [l] 
The sequence of prime numbers are on the horinzontal axis, and the vertical axis 
symbolizes the space of curves. The Katz-Sarnak approach is figured as a (horizon¬ 
tal) limit of vertical averages g p over vertical lines, and the Sato-Tate approach is 
a mean performed on horizontal lines. 

5. Expressions of the law of the trace in genus 2 

Assume now g = 2. Our purpose is to express the density of the distribution 
of the trace function r on USp 4 with the help of special functions. In order to 
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do this, the first step is to compute the characteristic function. The density of the 
Weyl measure on X 2 is 

62 ( 61 , O 2 ) = ^ sin 2 s ^ 2 ^ 2(2 cos 62 — 2 cos 6b) 2 . 

The fundamental alcove is 

a 2 = {(e 1 ,o 2 ) e x 2 | 0 < e 2 < e l < n}. 

The maximum of S 2 in A 2 is attained at the point 

dm = ( oim, 7T - a m ), where tan ^ = \] 2 + V%, 8(0 m ) = 

We have r o h(6 1; d 2 ) = 2 cos 6 1 + 2 cos 0 2 , and the characteristic function of r is 

f T (t)= [ e 2it ( cos 0 i + cos e As 2 (9 ll d 2 )d9i dd 2 ■ 

Jx 2 

Proposition 5.1. For every t £ K, we have 

? M _ 4Ji(2f) 2 6J\(2f)J 2 (2t) , 4J 2 (2t) 2 

■> T X> ~ V2 +3 + +2 


Here, J\ and J 2 are Bessel functions. 
Proof. Let 



V a (x) = 

2 5 cos 2 x sin 2 x = 

8 sin 2 ( 2 a:) 


V b (x) = 

2 5 cos 2 x sin 2 x cos 2 x = 

4 sin 2x cos 4a: 


V c (x) = 

2 5 cos 2 x sin 2 x cos 2 2 x = 

2 sin 2 4a; 

Then 





32it 2 5(x , y) 

= V c (x)V a (y) + V a (x)V c (y) - 

-2V b (x)V b (y). 

and 






F(t) = 2V a (t)V c (t) - 2 V b (t) 

2 

But 





Va(t) = ^ Ji(2t), V b (t) = ^ J 2 (2t), V c (t) = ^ M2t) - -^J 2 (2t), 

and the result follows. □ 


We now compute the moments M n (r) of r. By Proposition 5.1 the characteristic 
function can be expressed by a generalized hypergeometric series [HI §9.14, p. 1010]: 


f T (t) = iF 2 (^3,A-,-U 2 ) =£(-1) 


n—0 


, (j)n 

(3)rt(4)^ 


n\ 


where (a) n = a(a + 1)... (a + n — 1) is the Pochhammer’s symbol. It then follows 
from (3.6) that the odd moments are equal to zero. Since 


(|)n = 24 (n+±)T(n+±) 

(3)n(4)n \fri r(n + 3)T(n + 4) 


and [5] p. 897] 


r ( n+ - ) = Vtt 2 


- 2 n n )! 
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we obtain 


M 2 n (r) = 


6.(2n)!(2n + 2)! 


for n > 0. 


n\(n + l)!(n + 2 )!(n + 3)! 

One finds as expected Mihailovs’ formula, in accordance with 12, §4.1], which 
includes another formula for and also [T?» p. 126]. 

In what follows, four different but equivalent expressions for the distribution of 
t are given. 

5.1. Hypergeometric series. An expression of the density f T of the distribution 
of t is the following. Recall that Gauss’ hypergeometric series 

2 F l( a,6;c;,) = 5:^— - 

converges if |-| < 1 0 §9.1.0, p. 1005], 

Theorem 5.2. If \x\ < A, we have 

M x ) = 

This theorem immediately follows from the following lemma. 

Lemma 5.3. If \x\ < 4, we have 
64 


1 

/ z 2 \ 4 ^ , 

( 3 

5 r 

x 2 \ 


[ 1 — —— J 2-^1 



j; 1-) 

47T 

l 16 J 1 

12 ’ 

2 

’ 16 J 


f T {x) = —^m{xyl(m{x)), where m{x) = 1 — 


57T 2 


16’ 


and 


Moreover 


I(m) = t 2 


, 57T „ 

,{m) = 256 jFl 


1 ~t 2 
1 — mt 2 

3 5 


dt. 


2 ’ 2 


; 5; m . 


Proof. We use a formula of Schlafli, see [HI Eq. 1, p. 150]. If p and v are real 
numbers, then 

2 /' 7r / 2 

Jfj,(t)J u (t)=— / cos ip) cos(p — u)ipd(p (p + v>—l). 

n Jo 

As particular cases of this formula, we get 


Mt) 2 = - Ji hr 


ut 


du 






du 


\J 16 — u 2 


ut 

y 


du 


\/l 6 ^ 


By transferring these equalities in Proposition 5.1 we obtain 

Mt) = |y(2i) 2 - + ^J 2 (2t) 2 


2 
7r 


4 3w . . 4 

pMut) - ^3 J 3(^) + p2 M ut ) 


du 


\/l6 — u 2 
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and since 


we have 


l f 00 — 

f-r(x) = — / f T (ft) cos txdt, 


(51) = H vwhl 


4 3u . . 4 

pM ut ) - ypMut) + p Mut) 


cos txdt. 


We use now a formula of Gegenbauer on the cosine transform, see [151 p. 409] and 
[T51 Eq. 3, p. 50]. Assume Rei/ > —1/2, u > 0 and let n be an integer > 0. If 
0 < x < u, then 

[ t~ U J„ +2n {ut) COS tx dt = {-l) n 2 V ~ 1 U~ V , 

Jo T(2v + n) 

where C'/(x) is the Gegenbauer polynomial. If u < x < 00, this integral is equal to 
0. From Gegenbauer’s formula we deduce that if 0 < x < u, then 

1 ( u 2 — x 2 ) 3 / 2 




pOO 

/ t~ 2 J 2 (ut) cos txdt = 

Jo 

f°° -s 

/ t J 3 (ut) cos tx dt = 
Jo 

pOO 

/ t~ 2 J±(ut) cos txdt = — 

Jo 


3 u 2 

1 ( u 2 - x 2 ) 5/2 
15 u 3 
1 ( u 2 — x 2 ) 3 / 2 / 12 x 2 


30 u 2 


— 2 


since C 2 (x) = 12x 2 — 2. Transferring these relations in (5.1), we get 


57 r 2 / x (x) = 16 f 
J X 


4 (w 2 — x 2 ) 3 / 2 drt 


'u 2 Vl6 — u 2 


(m 2 — x 2 ) 5 / 2 dw 2 


r4 ( u 2 — x 2 ) 3 / 2 du 


\/l6 — u 2 ’ 


and this leads to 


5tt 2 J x M 4 


By the change of variables 


/- X 

it = 4 a/1 — m(x)t 2 , where m(x) = 1 — — ■ 


we obtain 


U(x) = 


64to(x) 


4 r 1 


t 2 


1-1 


2 \ 5 


dt , 


57 r 2 do \1 — m(x)t 2 / 
which is the first result. Euler’s integral representation of the hypergeometric series 
is 

r( c ) r 1 t b -\i-t) c ~ b - 1 


2 -Pi(a, b; c; z) = 


■ dt 


T(b)T(c-b)J 0 (1 -tz)° 
if Re z > 0, and Re c > Re b > 0. From this we deduce, with the change of variables 
t = u 2 , that 


57T 


J M = ^ 2F1 r, a;5; 


256 


3 5 


2 ’ 2 ’ 


which is the second result. 


□ 
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5.2. Legendre function. Another expression of f T is given by the associated Le¬ 
gendre function of the first kind IP^z), defined in the half-plane Rez >1. If a is 
not an integer > 1, and if b > 3/2, this function is defined by 0 Eq. 8.702, p. 959] 


nW = 


z + 1 


Fi I —6, b + 1; 1 — a: 


1 - ^ 


T(1 - a) \z — 1 / 

If a = m is an integer and if 2 > 1 is real, then [9J Eq. 8.711.2, p. 960] : 

7™(z) = ^ ^ a J (z + \J z 1 — 1 cos ipj cos rwpdip. 

If a = 0, this is the Laplace integral. 

Theorem 5.4. If |cc| < 4, then 


157T 

Proof. By Theorem |5.2[ we have 

1 


f T (x) = \f\x\ ( 1 - tf ) 


16 


x 2 + 16 
4x 


F(x) =—m(x) 2^1 5; m(x) I . 


47T 


3 5 


2 ’ 2 ’ 


But PJJ p. 51] 


Fi ( 5;z) =(l-z)~ 3 ^ 2 F 1 ( ^;3;- 


2 2 


3 7 {\-y/T=z? 


2 2 


4 y/1^ 


and [HI p. 47] 


3 7 


2 ’ 2 ’ 


Fi =(i-« -a,-; 3 ; 


1 3 


2 ’ 2 ’ 


On the other hand, if z = m(x), then 

(1 - yT ~zf _ (. x - 4) 2 

4a/1 - 2 16* 

By the definition of Legendre functions, 


Pf 2 


and this implies 


Since 


* — 4 
x + 4 


F( 1 3 .,. (*~ 4 ) 2 ’ 


1 /* 4 

2 V4 + x 




2 2 


1 / x 4 


16* 


T(6 — m + 1) 
if to £ Z, we obtain the required expression. 


Since 2 -Fi(a, 6; c; 0) = 1, we deduce from Theorem 5.2 that 
JVO) = 4 f 1 - + 0(x — 4) 5 


47T 


16 


and hence, in accordance with m P- 126]: 


□ 


4 
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Corollary 5.5. If |x| = 4 — e, with e — > 0 and e > 0, then 


Since 


fAx)= 647t + ° (£5) - ° 


lim sfxy\ + -) N ) = -- 

k->o 2 \2'4 x ) 7 r 


we deduce from Proposition |5.4| that the maximum of f T is reached for x = 0, and 


fA o) = 


64 

157T 2 


= 0.432 ... 


The graph of f T is given in Figure [2]; we recover the curve drawn in |121 p. 124]. 



5.3. Elliptic integrals. Another expression of f T is given by Legendre elliptic 
integrals. Let 


K(m) 


r /2 dy' 

Jo \/l — m sin 2 Lp 



be the Legendre elliptic integrals of first and second kind, respectively. The imple¬ 
mentation of /t in the Maple software gives: 


Corollary 5.6. If |x| < 4, then 
15 

— 7 r 2 f T (x) = (to 2 — 16to + 16)E(to) — 8 (to 2 — 3 to + 2 )K(m), 

64 

where m = 1 — (x 2 /16). □ 

The mention of the existence of such a formula is made in [7j. 
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5.4. Meijer G-functions. Another expression of f T is given by Meijer G-functions 
0 §9.3, p. 1032]. They are defined as follows : take z in C with 0 < |^| < 1 and 
to, 7 i, p, g in N. Then 


G: 


p,g 


CL\ ? , CLp 

bi,...,b q 


Uk=ins+b k ) nLirh-« t +i) z - Sds 


2i7T Ic nLn+l r ( S + «fe) ' nLm+1 r (- S - bk + !) 

Here, a\,..., a p , b ±,..., b q are a priori in C, and G is a suitable Mellin-Barnes 
contour. 


Corollary 5.7. If \x\ < 4, 


6 


16 


Mx) = -G( — I, with G(z) = G\ 


2,0 


5 7 

2 > 2 

0 , 1 


We have 


Giz)= lL 




Re s=c r (s + 2) r (s + 2 


with 0 < c < |. 


Proo/. If |z| < 1, then QU 07.34.03.0653.01]: 


G: 


2,0 


6 , —a + b + c 


= ^ (l Z ) a ~ b ~^ ~H-2a+2c-l)+fcfn- a + b +l 

T(a-6) ( j 

and the left hand side is equal to zero if \z\ > 1. Hence, if |z| < 1, 


G; 


2,0 


5 7 

2 ’ 2 

0 , 1 


(1 zfz 1,A *P 17 2 2 ( ) 


and we apply Theorem 5.4 


Corollary 5.8. If \x\ < 4, then the repartition function of t is 

„ 2 X j 
2 ’ 

with 


_ , s 3x f x 
= — G Th 

7T V 16 


G(z) = Gil z 


15 7 

2 ’ 2 ’ 2 
0 , 1 , -i 


Proof. According to [IP], 07.34.21.0003.01], we have 


1 f1171,71 

T p,q 


d \5 ... 5 dp 

bi,...,b q J 

By Corollary 1 5. 7[ a primitive of / T is 


az — ~ G p+l g+1 I 2 : 


4-o(*) = “ j 

and $o(—4) = — 1 / 2 . 




1 ) 7T 3 ’ 3 1 16 


1 cy, Q-l 5 ... 5 dp 

b \7 • • •, bq, G; 


15 7 

2 ’ 2 ’ 2 

0 , 1 , 


□ 


□ 
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5.5. The trace in SU 2 x SU 2 . In order to present a comparison with the above 
results, we give here without proof the distribution of the trace of a compact semi¬ 
simple subgroup of rank 2 of USp 4 , namely, the group SU 2 x SU 2 . If 


x = (xi, X2) 


the map 


and 


Tt{x) = 


-bi 

Q>i 

) e su 2 , 

\ a i\ 2 

+ M 2 = 1, 

/ Oi 

0 

-h 

0 ^ 


0 

a -2 

0 

-62 


bi 

0 

ai 

0 



b 2 

0 

0,2 / 



defines an embedding 

7T : SU 2 x SU 2 -USp 4 


* = 1 , 2 , 


whose image contains the maximal torus T of USp 4 . We put 


p(x) = Trace ir(x). 


The characteristic function of p is the square of the characteristic function of the 
distribution of the trace of SU 2 : 


f P (t) = 


Jim 2 

t 2 


The even moments are equal to zero, and the odd moments are 


M 2 n(p) — C n C n +i — 


2 (2n)!(2n +1)! 
(n!) 2 (n + l)!(n + 2)! 


where 


Cr, 



is the nth Catalan number. One finds the sequence 


1,0,2,0,10,0,70,0,588,0,5544... 
in accordance with the sequence A005568 in the OEIS m■ 
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Theorem 5.9. If |x| < 4, the density of the distribution of p is 


1 


2tt 


„2 \ 


f P (x) = — 1- — 2 Fi -;3;1 - — 


16 J 


1 3 


) \ 


2 2 


16/ 


Corollary 5.10. If \ x\ = 4 — e, with e -A 0 and e > 0, then 


r , N = e 2 g 3 £ 4 

p 87 t 647t 40967T 

The maximum of f p is reached for x = 0, and 


+ 0(e 5 ). □ 


f P { 0) = 

The graph of f p is given in Figure |4] 


8 

37T 2 


= 0.270... 


□ 



Figure 4. Density of the distribution of p. 


6. The Viete map and its image 

Another approach of the distribution of the trace is realized by an algebraic 
form of Weyl’s integration formula, using symmetric polynomials. This comes from 
a general program developed by Kohel PI, formerly outlined by DiPippo and Howe 
in [B], If t = (ti,..., t g ) £ C 9 , consider a monic polynomial 

( 6 . 1 ) h t (u) = (u -ti)...(u- t g ) = u 9 - 4 -f (-l) ff s 9 (t) 

in C[u]. Here 

Sn{t) = 'y ^ til ■ . ■ tik 

is the elementary symmetric polynomial of degree n in g variables. The discriminant 
of h t is 

( 6 . 2 ) disc ht = D 0 (t) = JJ(t* - tj) 2 . 

j<k 

The Viete map s : C 9 —> C 9 is the surjective polynomial mapping 

s(ti ,..., tg) (si(t),...,s 9 (t)), 
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where t = (ti ,..., t g ), inducing a bijection 

C 9/6 g ———> C 9 

which is a homeomorphism, because the map between the corresponding projective 
spaces is a continuous bijection between compact spaces. Hence, the Viete map is 
open and proper. We denote by 

n 9 = s(R 9 ) 

the closed subset which is the image of the Viete map. Hence, (si,..., s g ) £ n 9 if 
and only ht(u) has only real roots. The induced map 

r 9 / 6 9 — 21 —»• n g 

is a homeomorphism. The fundamental chamber of R 9 related to & g is 
C g = {t £ R 9 | ti < t 2 < ■ ■ ■ < t g } 

and C g is a fundamental domain for 6 g in R 9 . We are going to show that s is a 
local diffeomorphism at the points of an open dense subset of R 9 . For this purpose, 
we calculate J(s), where J(f) denotes the jacobian matrix of a polynomial map 
f : C 9 —> C 9 . Recall that the power sums 

Pn(t) = H-b tg (n > 1) 

can be expressed in terms of elementary symmetric polynomials. Precisely, from 
Newton’s relations 

n —1 

Pn = y^(-l y~ 1 s j p n -j + (-1 ) n ~ 1 ns n (n > 1), 
j =i 

we obtain p 3 J p. 28] : 


Si 

1 

0 

. 0 

2s 2 

Si 

1 

. 0 

ns n 

Sn— 1 

Sn-2 ■■ 

• Si 


This is related to a more suitable expression [5J p. 72], [2J Ch. IV, § 6 , Ex. 6 ], 
obtained by Albert Girard [ 8 J in 1629, and sometimes attributed to Waring (1762): 


Proposition 6.1 (Girard’s formula). If 1 < n < g and s = (si,..., s g ), let 

ru i u i i u i m 

V n (s) =n 




(b 1 + b 2 + --- + b g -l)\ bl b 


where u n = (— 1 )" 1 s n for 1 < n < g, and the summation being extended to 


Then 


d 5 n — {b — (bi,..., bg) £ N 9 | bi + 2b 2 + • • • + gb g — n} . 


Pn=V n ° S. □ 


The map ip <—>■ ip o s defines an isomorphism 

s* :Z [si,...,Sg] — > Z[ti,...,t g ] s y™. 

Since D 0 £ Z[ti,..., t g ] sym , there is a polynomial do £ Z[si,..., s g ] such that 

(6.3) d 0 (s(t)) = D 0 (t) = - tj) 2 . 

j<k 








TRACE IN THE UNITARY SYMPLECTIC GROUP 


21 


Let 


Ug — {t £ 


D 0 (t)^0}, n° = {s e 


d 0 (s) ^ 0} . 


Then 11° = s(f/ g ), and 11° is a dense open set of II g . The roots of the polynomial 
h t £ K[u] given by (6.1) are real and simple if and only if s (t) £ 11°. 


Proposition 6.2. With the preceding notation: 

(i) If t £ R 9 , then 

|det J(s)(t)| = y/D 0 (t) = - tj\. 

j<k 

(ii) The map s is a local dijfeomorphism at every point of U g . 

(iii) The map s is a dijfeomorphism from the fundamental chamber C g to 11°. 


Proof. Define two polynomial maps from C 9 to C 9 : 

P(t) = (pi(t),---,Pg(t)) and v{s) = (v 1 (s),...,Vg(s)). 

Then p = v o s by Girard’s formula |6.1| If 1 < n < g, then 

v n (s) = (—1 ) n+1 ns n + v' n (s), 

where v' n (s) depends only of S\,... , s„_i. This implies that J{y) is lower triangular, 
with n-th diagonal term equal to (—1 ) n+1 n. Hence, 

det J(v) = (—l)l9/ 2 l g\ 

On the other hand, 



( 1 

1 

1 \ 

J(p) = 

kt\~ x 

ktt 1 ... 

ktt 1 


\gtr l 

gtr 1 ■ ■ ■ 

g^u 1 ) 


Then J(p) = D.V(t), where D is the diagonal matrix diag(l, 2 ,g), and 

1 \ 

tn 

ti- 1 ) 


V(t) = 


( i 

tl 

vr 1 


is the Vandermonde matrix. Hence, 


1 

t2 

+ 9-1 


det J(p) = g\ det V ( t ) = g! (*fc ~ t o)^ 

j<k 


and since J(p) = J(v).J(s), we get |I]), which implies (|TI|). Then (|m| comes from 
the fact that s is injective on the open subset C g of U g . □ 


The bezoutian of ht is the matrix 



( Po 

Pi ■ 

■ Pg -i I 


B(t) = V{t)}V{t) = 

Pi 

P2 ■ 

Pg 

G M s (R) 


\Pg -1 

Pg ■ 

■ P2g-2 / 



in such a way that det B(t) = D 0 (t). 


Lemma 6.3. Let h t £ R[w]. The following conditions are equivalent: 













22 


GILLES LACHAUD 


(i) The roots of h t are real and simple, i.e. s(t) £ 11°. 

(ii) The bezoutian B(t) is positive definite. 


Proof. This is a particular case of a theorem of Sylvester, which states that the 
number of real roots of h t is equal to p — q, where ( p , q ) is the signature of the real 
quadratic form 

Q(x) = t x.B(t). x, 

where x = (xo, • • •, %-i) £ R 9 - Here is a short proof: if 1 < j < g, define the linear 
form 

Lj(x) = £ 0 + X\tj + • • • + Xg-it^ 1 . 

Then x.V(t) = (Li(x ),..., L g (x)), and 

Q{x) =J2 L j( x ) 2 - 

j =i 

If tj £ R, the linear form Lj is real. If t 3 ^ K, the non-real linear form Lj = A :j +iBj 
appears together with its conjugate, and 

L] + L] = 2A]-2Bj. 

This shows that if ht has r real roots and s couples of non-real roots, the signature 
of Q is (r + s, s). □ 


The bezoutian B{t) is positive definite if and only if its principal minors 

(1 < j < g) 




Po 

Pi ■ 

• Pj -1 

Mj (pi, • 

■■>Pg) = 

Pi 

P2 ■ 

Pj 



Pj-i 

Pj • 

■ P2J-2 


are > 0, see for instance [3], Prop. 3, p. 116]. By substituting to the power 
sums their expression given by Girard’s formula of Proposition |6.1[ we obtain g 
polynomials 


<,j = Mjov eZ[s 1 ,...,s g \ (1 <j<g). 


Of course, m\ = g , and 


Mg{pi,...,p g ) = det B(t) = D 0 (t), 


hence, m g (s) = do(s). As a consequence of Lemma 6.3 we obtain 
hence: 


H° = {, £ 


mfis) > 0 if 2 < j < g} , 


Lemma 6.4. We have 

n g = {s £ 


\j(s) > 0 if 2<j <g} . □ 


Example 6.5. If g = 2, then do(s) = sf — 4 s 2 , and 

n 2 = {s £ K 2 | do(s) > 0 } . 

Example 6 . 6 . If g = 3, then 

do(s) = s\s\ — 4 — 4 sfs 3 + I 8 S 1 S 2 S 3 — 27 S 3 , 

and m 2 (s) = 2(sf — 3s 2 )- But if d 0 (s) > 0, then m 2 > 0. Actually, if 


p = 


sf - 3 s 2 


q = 


2s 3 — 9sis 2 T 27 s 3 
27 
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then 

do(s) = -(4 p 3 + 27 q 2 ), m 2 = -6 p. 

If d 0 (s) > 0, then 4 p 3 < —27 q 2 and p < 0. Hence, as it is well known, n 3 is defined 
by only one inequality: 

n 3 = {s e i 3 | do(s) > o}. 


7. The symmetric alcove 


The symmetric alcove is the compact set 

^ g = s (Ig) C big- 

We have s(Ag) = s (I g ), and the induced map 


I9/G9 


-> So 


L g/^g r ^g 

is a homeomorphism, leading to the commutative diagram 



If p £ C[£i,..., t g \ is a symmetric polynomial and if A £ C, define 
P( A; t) = p(X + h,..., A + t g ), t = (ti,.. .,t g ). 

The polynomial p( A; t) is symmetric with respect to t. 

Lemma 7.1. If t £ and A > 0, the following conditions are equivalent: 

(i) Sj(A; t) > 0 and s,(A; —t) > 0 for l < i < g. 

(ii) N < A for 1 < i < g. 


Proof. It suffices to prove the following result: if t £ R 9 , the following conditions 
are equivalent: 


(i) Si(t) > 0 for 1 < i < g. 

(ii) ti > 0 for 1 < i < g. 

, namely 


If h t £ K[u] is defined as in (6.1 


h t (u) = (u - ti)... (u - t g ) = u 9 - Siu 9 1 H-1- (-l)®s g , 


let f^(u) = (—1) 9 /(— u). Then 


f b (u) = [u + ti) ... {u + t g ) = u 9 + Siu 9 1 + s 2 u 9 2 4-+s g , 

and if 0 is satisfied, the roots of (—1) 9 /(— u) are < 0, and this implies |h]). The 
converse is trivial. □ 


The polynomial Si(A;f) £ C[ti,... ,t g ) is a linear combination of elementary 
symmetric polynomials of degree < i: 
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Lemma 7.2. If 1 < i < g and if A > 0, then 

Si(X;t) = 

with 


L+(\-,s) = Y / 


k=0 


g - i + k 
k 


Si- k X k 


which is a linear form with respect to si,..., Sf. Similarly, 


Si{ A; -i) = ^ (A;s (t)), 


where 

L~ (A; si, s 2 ,..., Si) = L+(A;-si,s 2 ,---,(-l)*Si). 
Proof. The Taylor expansion of ht(u — A) with respect to u shows that 


Si (A, £) 


(- 1 ) 


j ^(-A) 

( 5 -*) ! 


The first formula is obtained by transferring these equalities in the Taylor expansion 
of h[ 9 ~ l \ A) at 0 : 


A) = ^(-l) fc ^- i+fc) (0)^ . □ 

fc =0 

The second formula is deduced from the first by noticing that Si(—t) = (—1 ) l Si(t). 
Considering that s 0 = 1, we have 
si (A; t) = si(t) + g\, 

s 2 (A; t ) = s 2 (t) + (g — l)Asi(t) + ^—- A 2 , 

s g { A; t) = Yl s 9-k(t)X k = (—l) 9 /q(—A) = P(L + A). 

k —0 i =1 

Hence 

Lf( 2 ;s) = ±si + 2 g, Lj( 2 ;s) = s 2 ± 2 ( 3 - l)si + 2g{g- 1 ), 

and 

(7.1) Lf(2-s) = J2^Sg-k, L s -(2; S ) = ^(-l) 9 - fe 2 fc Sff _fc- 

k—O k -0 

From Lemmas 17. 11 and 17.21 we obtain 

Lemma 7.3. Assume t £ R 9 . Then t £ I g if and only if s(t) £ Q g , where 
0 g = {s £ R 9 | Lf( 2; s) > 0 for 1 < i < g} ■ □ 

Notice that the polyhedron 0 g is unbounded. 

Theorem 7.4. //E g = s (I g ), then 

^9 = ^ 
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where II g and 0 g are defined in Lemmas 6.f and 7.3 Moreover, T, g = s(A g ) is a 


semi-algebraic set homeomorphic to the g-dimensional simplex, and 


s r 


e 9 c n 


-2 l 


Proof. By definition, II g = s(R s ), hence, the first statement follows from Lemma 
7.3 The properties of £ g follow from Proposition m and the last statement is 


just a consequence of the definition of si(t),..., s g {t). 


□ 


Example 7.5. If g = 2, then do(s) = sf — 4s2, and 

n 2 = {s e R 2 | d 0 (s) > o}, 

as we saw in Example |6.5| The triangle 0 2 is defined by four inequalities: 

Lf(2,s) =± Sl +4> 0, 

L±(2, s') — S 2 i 2si + 4 > 0. 

The symmetric alcove S 2 the curvilinear triangle, drawn in Figure [5j contained in 
the square [—4,4] x [—4,4]. 



Example 7.6. If g = 3, then 

d 0 (s) = sjs 2 — As\ — 4sfs 3 + 18 sis 2 S3 — 27 s 3 , 


and 


n 3 -{se R 3 | do(s) > o}, 
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as we saw in Example |6.6| The polyhedron ©3 is defined by six inequalities 
2, s) = ±Si + 6 > 0, L*2 (2, s) = s 2 ± 4s 1 + 12 > 0, 

T^~(2, s) = AS3 -F 2s 2 A 4si + 8 > 0. 

The intersection of ©3 and of the box 

[-4,4] x [-12,12] x [-8,8] 
make up a polytope P3 with 6 vertices 

Pi = (-6,12, -8) = s(—2, —2, —2), P 2 = (—2, —4,8) =s(2,-2,-2), 
P 3 — (2, —4, —8) = s(2,2, —2), 7,4 = (6,12,8) =s(2,2,2), 

Ps = (6,12,-8), P6 = (-6,12,8), 


and 7 facets supported the following hyperplanes: 

s 2 = 12, s 3 = ±8, if (s) = 0, Lf(s) = 0. 


Then 


S3 — 1I3 n P3. 

The symmetric alcove S3 is drawn in Figure[6] This set is invariant by the symmetry 
(si, s 2 , S3) 1—>■ (—Si, s 2 , —S3). The graphical representation leads to suppose that 

S3 = II3 D A3, 


where A3 is the tetrahedron with vertices Pi, P2, P3, P4 and support hyperplanes 

L f(s)= 0 , Lq(s) = 0, 
where Lq (s) = 24 ± 4si — 2s 2 3s3. 


8. Symmetric integration formula 
The map tp <—> tp o s defines an isomorphism 

s* : ^g) —^ ^(/ ff ) sym = ^(4) 

If F G ^(G) 0 , we denote by F the unique function in < ^(/ £ ,) sym such that 

F o s (t) = F o k(t), 

that is, 

F o s(2cosdi,..., 2 cos 0 9 ) = F o h( 0 i,.. . ,6g). 

Then the map F i-a F is an isomorphism 

nor —^ ^(s 9 ). 


inducing by restriction an isomorphism, cf. Proposition A.l in the appendix: 

R(G) - > Z[si,..., s g ]. 

Since D\ G Z[fi,..., f g ] sym , there is a polynomial di G Z[si,..., s 9 ] such that 


( 8 . 1 ) 

hence 


di(s(f)) =T>i(t) = 

l=i 

Di (t) = Sg(2; t)Sg(2; —t), 






TRACE IN THE UNITARY SYMPLECTIC GROUP 


27 



where s±(A;t) is defined in Lemma [7.2| and 

d l (s) = L+(2-s)L-(2-s), 
where L± (2; s ) is defined by (Hill- 

Proposition 8.1 (Symmetric integration formula). If F € G)°, then 


F (m) dm = / F (s)p g (s)ds, 


with ds = ds i... ds g , and 


Vg(s) = 


where do(s) is given by (6.3) and di(s) by (8.1). 


Proof. By Proposition 6.2 we can perform a change of variables from E 
apart from null sets, by putting s = s (t). If tp £ we have 

ds 




\J do(s) 


J An 


dt. 



g\ f ip{s{t))X g (t). 

d Ag 


g tO Ag, 


This implies 
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If ip = F, then 


F (s)ug(s) ds = g\ / F o k(t)\ g (t) dt = / F o k(t)\ g (t) dt, 


the second equality by using (3.5). One concludes with the help of Proposition 

m □ 


In other words, if (j> g is the characteristic function of E 9 , the function v g (j) g is the 
joint probability distribution density function of the distribution for the random 
variables Si,... ,s g . 

If t is the trace map on USp 2g , then 

t(s) = si. 

One obtains an integral expression of the density by the method of integration 
along the fibers already used in Remark |3.5[ which reduces here to an application 
of Fubini’s theorem. The linear form s i-> si is a submersion from the open dense 
subset U = s(A s ) of E 9 onto J = (—2 g, 2 g), and if x € J, then 

V x = {sGU | si = a;} 


is just an intersection with a hyperplane. If 

a x (s 2 , ...,s g ) = v g {x, S 2 ,...,s g )ds 2 A • • • A ds g , 

then 


-(z) = / v g {s)ds\ A • • • A ds g = / dx a x (s 2 ,..., s g ), 
J si<z J—2g JVs 


hence: 


Proposition 8.2. If |x| < 2 g, then 

U(x ) = 


^x(s2? ■ ■ • j S 9 ). fH 


Example 8.3. If g = 2, the symmetric alcove E 2 is described in Example [ 73 ] Here, 

do(s) = sf - 4s 2 , di(s) = (s 2 + 4) 2 - As\, v 2 {s) = y/d 0 (s)di(s). 

The graph of v 2 is shown in Figure [ 7 ] The maximum of v 2 in E 2 is attained at the 
point 

4 8 

s° = (0,with ^ 0 ) = ^-=0.155... 


By Proposition |8.2[ we find a definite integral : if x > 0, 

M x ) = ^2 J 4 [((y + 4 ) 2 - 4a;2 ) ( x2 - A y )} 1 d v- 

It can be verified that this formula is in accordance with Theorem 15.41 
Example 8.4. If g = 3, the symmetric alcove E 3 is described in Example |7.6| Here, 
do(s) = s 2 s 2 — 4s 2 — 4s 3 s 3 + 18siS 2S 3 — 27s 3 , 

di{s) = (2s2 + 8 )“ — (4si + S3) 2 , 

M s ) = ^3 \/do(s)di(s), 
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and 

14 = {s £ l 3 | do(s) > 0, ±S 3 + 2 s -2 ± 4x + 8 > 0} . 

The density is 

fA x ) = / v 3 {x,s 2 ,s 3 )ds 2 ds 3 . 

4 

With this formula in hands, we are able to compute the even moments : 

1,1,3,15,104,909,9 449,112 398,1489 410,21562 086 ... 

This sequence is in accordance with the results of [12 Sec. 4] and the sequence 
A138540 in the OEIS [16]. Actually, it is faster to compute this sequence by noticing 
that, according to Weyl’s integration formula of Proposition [372] the characteristic 
function of r is given, for y £ R, by 

fAv) = ^3 J D 0 (t)y/Di(t) cos(y(ti +t 2 +t 3 ))dt 1 dt 2 dt 3 . 

An implementation of this integral in the Mathematica software gives 


fAv) = 


24 - 


4J 1 (2 2/ ) 3 HJ 1 (2y) 2 J 2 (2y) 2(3 + y 2 ) Ji(2y)J 2 (2y) 2 , 5J 2 (2y) 


+ 


and it suffices to apply (3.6 1 to obtain the moments. An approximation of f T 
to any order in L 2 ([— 6, 6], dx) can be obtained from the sequence of moments, 
using Legendre polynomials, which form an orthogonal basis of L 2 ([—1,1], dx). For 
instance the maximum of f T is reached for x = 0, and we find 

/ T (0) = 0.396 467... 

The graph of f T obtained by this approximation process is drawn in Figure [8] 


As a final instance, we come back to the case g = 2 and apply the symmetric 
integration formula to the distribution of the character t 2 of the exterior power 
A 2 7 t of the identity representation 7 r of USp 4 on C 4 , namely 

t 2 o h(9) = 2 + 4 cos cos0 2 , (s) = s 2 +2. 
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Figure 8. Density of the distribution of r, case g = 3 


The density of T 2 is given by 

f To (x) = f \/l 6 + 8x + x 2 — 4 z 2 \/z 2 — Ax dz , 

Ji+ 


with 


I+= (~ 


x + 4 x + 4 
2 ’ 2 
x + 4 


, —2 yfx ) U 2^/x, 


x + 4 


if — 4 < x < 0, 
if 0 < x < 4. 



The implementation of this integral in the Mathematica software gives the se¬ 
quence of moments (see below), from which one deduces: 

Proposition 8.5. Assume |rr | < 4. Then f-r 2 {x) is equal to 

(x(A - 2te + 16)E (l - 1?) + 4(3A - 8x + i8)K (l - i?)) . □ 


24t r 2 
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The maximum of f T2 is reached for xq = —0.605..., and f T2 (x o) = 0.403... 
Moreover 

/^ (0) = ^= 0 - 2 7°... 

This function is continuous, but the derivative has a logarithmic singularity: 


/t 2 0) 


\ogx 2 


0 . 


The graph of f T2 is shown in Figure [9] The moments M n of f T2 are obtained by 
numerical integration: 

1, -1,2, -4,10, -25,70, -196,588, -1764... 

Hence, the random variable r 2 has mean —1 and variance 2. This sequence is, up 
to sign, the sequence A005817 in the OEIS HU, such that 


A72n('T2) — C n C n+1 , M 2n +l — _ G, 


n+15 


where 


C n = 


n + 1 


is the nth Catalan number. 


Remark 8.6. The representation A 2 7 t is reducible, cf. for instance Lemma A.6 in the 
appendix. Actually, the two fundamental representations of USp 2 are the identity 
representation n = tti with character t and a representation 7 r 2 of dimension 5 and 
character \ 2 satisfying 

X 2 ° h(9) = 1 + 4cos0i cos0 2 , X 2 ( s ) = s 2 + 1. 

The representation 7 t 2 is equivalent to the representation corresponding to the mor¬ 
phism of USp 2 onto SO, 5 . Since t 2 = x 2 + 1, we have 

A 2 7T = 7T 2 ® 1, 


and t 2 (s) = s 2 + 2: this relation is an instance of Theorem A.7 in the appendix. 


The random variable x 2 , with values on [—3,5], is standardized, by Remark 3.4 
The moments of f Xo are 

0,1,0,3,1,15,15,105,190,945... 
in accordance with the sequence A095922 in the OEIS [SI- 

APPENDIX A. The character ring of G 

The character ring R(G ) of G = USp 2ff is the subring of ^(G ) 0 generated, as a 
Z-module, by the characters of continuous representations of G on finite dimensional 
complex vector spaces. Since every representation of G is semi-simple, the Z- 
module R(G) is free and admits as a basis the set G of characters of irreducible 
representations of G : 

R(G) = Zr - 

reG 

The virtual characters are the elements of R(G), and the characters correspond 
to the additive submonoid of sums over G with non-negative coefficients. The 
functions 9 i-a e*^(l < i < g) make up a basis of the discrete group T 9 , and if h{9) 
is as in (2.1 1 , the map h* : f i —> f o h defines an isomorphism from the group X(T) 
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of characters of T to the group X(T 9 ). Hence, if Z[X(T)] is the group ring, we have 
a ring isomorphism 

h* : Z[X(T)] ———> Z[{e Wj ,e~ iSj }}. 

Let Z[X(T)] W be the subring of elements invariants under W. Recall that the 
restriction map 

R(G) ———> Z[X(T)] W 

is a ring isomorphism [4j Ch. 9, §7, n°3, Cor., p. 353]. From the structure of W, 
we deduce that h* induces a ring isomorphism 

(A.l) R(G) - ~ - > Z[2 cos 0i,... 2cos 0 g ] sym . 

and, by putting (2 cos 9 1 ,... 2 cos 9 g ) = (ti,..., t g ), a ring isomorphism 

(A.2) R(G) — ^ 

On the other hand, the application p i—> ip os, where s is the Viete map, induces 
the classical isomorphism 

s* :Z[ Sl ,...,a g ] — > Z[t 1 ,...,t g ] B y m . 

If F € R{G), we denote by F the unique polynomial in Z[si,..., s g ] such that 

F o s(2cos0i,..., 2cos0 9 ) = F o h(9\, ..., 9 g ). 

We obtain: 


Proposition A.l. If G = USp 2s , the map F h-> F is a ring isomorphism 


R(G) —>■ Z[si,...,s ff ]. □ 

Recall from Remark 

degree 2 g in C[u] with all roots on the unit circle. We write a typical element of 
$ 2 g as 

2 g 

Pa{u) = y^(—1 ) n Cl n U 29 ~ n , 
n —0 

where a 2 g - n = a n for 0 < n < g. Moreover p a (u) = u 2g p a (u~ 1 ). The roots of p a 
come by pairs : if p a is monic, then 

Pa{u) = - e l6j )(u - e~ z9j ) = J|(u 2 - utj + 1), 

j = 1 i =1 

with tj = 2 cos 9j , and the coefficients a„ are symmetric polynomials in the variables 
{e l 0 3 ,e — *^}, invariant under conjugation. 


2.1 that $23 is the set of monic palindromic polynomials of 


Theorem A.2. 

relation 


If t = £ C 9 , and if 0 < n < 2 g, define a n (t ) by the 

a 2 g 

- utj + 1 ) = ^(-l) n a„(f)it 29-n . 

j =1 n =0 


If 0 < n < g, then 



where Soft) = 1 and s n (t) is the elementary symmetric polynomial of degree n. 









TRACE IN THE UNITARY SYMPLECTIC GROUP 


33 


We deduce Theorem |A.2| from the following lemma. 

Lemma A.3. If s = (so,..., s g ) G C 9+1 , let 

a 

h s (u) = ]T(-1 ) n s n u 9 ~ n , 

n—0 

and for 0 < n < 2 g, define q n {s) by the relation 

2 g 


g h s (u + u = ) n q n (s)u 29 71 . 


If 0 < n < g, then 


Proof. We have 


n/2 

dn{s) = Y 

3=0 


n—0 

g + 2j - n 
j 


3 n-2j- 


g h(u + u X ) = Zi 9 ^(—l) fe Sfc(M + M 1 ) 


-l\g-k 


k =0 


Since 

g-k 

U 9 (u + U~ l ) 9 ~ k = U 9 Yj 
j =0 

one finds 


g-k 

j 


g-k 

{u- x y- k - j u j = Y 

3=0 


g h(u + u~ 1 ) = Y C" 1 )* 


g-k 

3 


s k u 


g-k 

3 

k+2j 


k+2j 


j-\-k<g,j> 0,fe>0 

Let k + 2j = n. Then n runs over the full interval [0, 2 g] and 

j > 0 and k > 0 and j + k < g •<=>• j > 0 and 2 j < n and j > n — g. 
Hence, if 1 < n < 2g, we have 

i/2 

/ q + z .7 -n \ 

I $n—2j 5 


Qn(s) = Y 

j=max.(0,n—g) 

and the result follows. 


g + 2 j -n 
3 


□ 


If p is a Weil polynomial and if p(u ) = u 9 h(u + u 1 ), then h has real roots and 
is called the real Weil polynomial associated to p. 

Proof of Theorem \ A.ty In Lemma |A.3[ assume that 

g 

hs(u)= 

3 =1 

Then s n = s n (t ), where t = (fi,... ,t g ), and 

g 

u 9 h s (u + u~ x ) = J^[(zt 2 — utj + 1). 


3 = 1 


Hence, a n (t) = q n ( 1, Si(f), s n (t )). 


□ 
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Define an endomorphism q of C 9+1 by 

q : (so, •••, S g ) (q 0 (s),. .. , q g (s)). 

By Lemma the square matrix of order g + 1 associated to q is unipotent and 
lower triangular, with coefficients in N. For instance: 

(A.3) if g = 2 : q = 

Remark A.4. A reciprocal of the map Q : h K»• u 9 h(u + u -1 ) is constructed as 
follows. If n > 1, let T n (u) be the ro-th Chebyshev polynomial [9] p. 993], and 
c n {u ) = 2T n (u/2), in such a way that 

c n (u + m -1 ) = u n + u~ n if n > 1. 

Moreover put co(u) = 1. If p a £ 4> 2 g as above and if 

3 

[R.p](u) — 'y ] ( 1) a n c g - n (u), 

n =0 

it is easy to see that Q o R(p) = p. 



if 9 = 3 : q = 


/I 0 0 0\ 
0 10 0 
3 0 10 
\0 2 0 1 / 


Remark A.5. It is worthwile to notice that the map pi-xpoq defines an isomor¬ 
phism of the two ring of invariants: 

q* : Z[«i,..., a g ] —Z[ Sl ,..., Sg ] 

where we identify Z[ai,..., a g ] and Z[ai,..., a 2 g-i}/((a 2 g-n ~ o, n )). If we define a 
polynomial mapping a : C 9 —> C 9+1 by 

a :t = (fi,. ..,t g ) H, (a 0 (t),.. .,a g (t)). 

and if s(f) = (so(t), si(t),..., s g (t)) is the (extended) Viete map, then 


a = q o s. 

These maps are gathered in the following diagram: 




-AZ[fi,...,f s ] 


w 


R(G ) : Z[ai, ...,a g ] -> Z[si,..., s g ] — L -> Z[fi, ...,t g ] 


sym 


We apply the preceding to the character of the n-th exterior power A" 7 r of the 
identity representation 7 r of G in C 2 ®. For 0 < n < 2g, let 

(A.4) r n (m) = Trace(A ra to). 

Generally speaking, the representation A n 7 r is reducible, and we describe now its 
decomposition. For each dominant weight u of USp 2ff , we denote by tv(u>) the 
irreducible representation with highest weight ui, cf. [3]. The following lemma is 
used in Lemma 14.41 

Lemma A. 6 . Let ui\,... ,ui g be the fundamental weights of USp 2g . Then 
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(i) If 1 < 2 n + 1 < g, we have 

A 2n + 1 7T = (J) 7 t(W2j+i). 

0<j<n 

(ii) If 2 < 2n < g, we have 

A 2n 7r = 1 © 7 t(w2 j)- 

l<j<n 

Proof. See El Lemma, p. 62]; the corresponding result for a simple Lie algebra of 
type C g is proved in jj] Ch. 8 , §13, n°3, (IV), p. 206-209]. □ 


The characteristic polynomial of to € USp 2g is 

2 g 

cp m (u) = det(w. I — to) = y^(—1 ) n T n {m)u 2a ~ n . 

n=0 

The dual pairing 

AT x A 2g ~ n V -» A 2g V = C 

implies that T 2 9 _ n = r„ for 0 < n < 2 g, and this proves that cp m £ $ 2 g- If to is 
conjugate to h{6 \,..., 8 g ), then 

g 

CP m(u) = n<V 2 ~ ut j + !). 

1=1 

with tj = 2 cos 6j, and hence r n o k £ Z[t 1 ,.. -Ig] sym as expected. 

In the notation of Theorem |A.2[ we have 

(A.5) r n ok(t) = a n (t), 

and we deduce from this theorem: 


Theorem A.7. Let m £ USp 2g be conjugate to k(ti ,..., t g ). If 0 < n < g, then 


T 


n 



Sn—2j iff 


□ 


For instance, according to (A.3): 


if g = 2: t 2 (to.) = S 2 (t) + 2 (cf. Remark | 8 ii| , 

-if g = 3: t 2 (to) = s 2 (t) + 3, r 3 (m) = s 3 (t) + 2s]_(t). 
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